Towards Training Graph Neural Networks with Node-Level Differential Privacy

Graph Neural Networks (GNNs) have achieved great success in mining graph-structured data. Despite the superior performance of GNNs in learning graph representations, serious privacy concerns have been raised for the trained models which could expose the sensitive information of graphs. We conduct the first formal study of training GNN models to ensure utility while satisfying the rigorous node-level differential privacy considering the private information of both node features and edges. We adopt the training framework utilizing personalized PageRank to decouple the message-passing process from feature aggregation during training GNN models and propose differentially private PageRank algorithms to protect graph topology information formally. Furthermore, we analyze the privacy degradation caused by the sampling process dependent on the differentially private PageRank results during model training and propose a differentially private GNN (DPGNN) algorithm to further protect node features and achieve rigorous node-level differential privacy. Extensive experiments on real-world graph datasets demonstrate the effectiveness of the proposed algorithms for providing node-level differential privacy while preserving good model utility

Comparison study of raindrop size distribution in three regions of Shandong Province

To study the microphysical characteristics of precipitation in different regions of Shandong province-the Yellow River Delta, inland region and coastal region, the raindrop spectrum data at three sites (Kenli, Pingyin, and Jiaonan) from 2017 to 2020 were used to compare the raindrop size distribution (DSD) characteristics of different rain rates (R) and rainfall types (stratiform and convective rainfall). The results are as follows. (1) In 6 rain rate levels at 3 sites, there were certain degree of differences in the DSD spectral, the spectra width of mean DSD, and the contribution of different precipitation levels to the total precipitation amount. (2) In stratiform precipitation, the average DSD characteristics of Pingyin and Jiaonan were comparable to each other, but differences were found in convective precipitation at these 3 sites. (3) The distribution of the normalized Gamma spectral intercept parameter (log10Nw) and the mass-weighted average diameter (Dm) showed that, log10Nw of Kenli station was the lowest during stratiform precipitation. While the comparison with other national and international stations demonstrated that the distribution of log10Nw versus Dm from Pingin agreed with the observations in Seoul, Korea. (4) The fitting curves of the shape parameter (μ) and slope parameter (λ) from the Gamma distribution exhibited localized characteristics. According to the trend of μ and λ with R at all 3 sites, R=10 mm·h-1 can be used as a criterion for precipitation classification. At all three sites, when R increased greater than 100 mm·h-1, the values of μ remained nearly constant towards 1~2, while the values of λ tend to 2. (5) The fitting results between the reflectivity factor (Z) and R (Z=ARb) at 3 sites indicated that, the typical Z-R fitting formula (Z=300R1.40) for weather radars overestimated the precipitation of Kenli and underestimated the precipitation of Jiaonan. The values of A and b in the Z-R fitting formula also exhibited differences between convective and stratiform precipitation

Private Semi-supervised Knowledge Transfer for Deep Learning from Noisy Labels

Deep learning models trained on large-scale data have achieved encouraging performance in many real-world tasks. Meanwhile, publishing those models trained on sensitive datasets, such as medical records, could pose serious privacy concerns. To counter these issues, one of the current state-of-the-art approaches is the Private Aggregation of Teacher Ensembles, or PATE, which achieved promising results in preserving the utility of the model while providing a strong privacy guarantee. PATE combines an ensemble of "teacher models" trained on sensitive data and transfers the knowledge to a "student" model through the noisy aggregation of teachers' votes for labeling unlabeled public data which the student model will be trained on. However, the knowledge or voted labels learned by the student are noisy due to private aggregation. Learning directly from noisy labels can significantly impact the accuracy of the student model. In this paper, we propose the PATE++ mechanism, which combines the current advanced noisy label training mechanisms with the original PATE framework to enhance its accuracy. A novel structure of Generative Adversarial Nets (GANs) is developed in order to integrate them effectively. In addition, we develop a novel noisy label detection mechanism for semi-supervised model training to further improve student model performance when training with noisy labels. We evaluate our method on Fashion-MNIST and SVHN to show the improvements on the original PATE on all measures

Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis

Tensor factorization has been demonstrated as an efficient approach for computational phenotyping, where massive electronic health records (EHRs) are converted to concise and meaningful clinical concepts. While distributing the tensor factorization tasks to local sites can avoid direct data sharing, it still requires the exchange of intermediary results which could reveal sensitive patient information. Therefore, the challenge is how to jointly decompose the tensor under rigorous and principled privacy constraints, while still support the model's interpretability. We propose DPFact, a privacy-preserving collaborative tensor factorization method for computational phenotyping using EHR. It embeds advanced privacy-preserving mechanisms with collaborative learning. Hospitals can keep their EHR database private but also collaboratively learn meaningful clinical concepts by sharing differentially private intermediary results. Moreover, DPFact solves the heterogeneous patient population using a structured sparsity term. In our framework, each hospital decomposes its local tensors, and sends the updated intermediary results with output perturbation every several iterations to a semi-trusted server which generates the phenotypes. The evaluation on both real-world and synthetic datasets demonstrated that under strict privacy constraints, our method is more accurate and communication-efficient than state-of-the-art baseline methods

Strand antagonism in RNAi: an explanation of differences in potency between intracellularly expressed siRNA and shRNA

Strategies to regulate gene function frequently use small interfering RNAs (siRNAs) that can be made from their shRNA precursors via Dicer. However, when the duplex components of these siRNA effectors are expressed from their respective coding genes, the RNA interference (RNAi) activity is much reduced. Here, we explored the mechanisms of action of shRNA and siRNA and found the expressed siRNA, in contrast to short hairpin RNA (shRNA), exhibits strong strand antagonism, with the sense RNA negatively and unexpectedly regulating RNAi. Therefore, we altered the relative levels of strands of siRNA duplexes during their expression, increasing the level of the antisense component, reducing the level of the sense component, or both and, in this way we were able to enhance the potency of the siRNA. Such vector-delivered siRNA attacked its target effectively. These findings provide new insight into RNAi and, in particular, they demonstrate that strand antagonism is responsible for making siRNA far less potent than shRNA

Partial carbonization of aramid fibers

M.S.S.B. Warne

A familiar peer improves students’ behavior patterns, attention, and performance when learning from video lectures

Abstract Synchronous online learning via technology has become a major trend in institutions of higher education, allowing students to learn from video lectures alongside their peers online. However, relatively little research has focused on the influence of these peers on students’ learning during video lectures and even less on the effect of peer familiarity. The present study aimed to test the various effects of peer presence and peer familiarity on learning from video lectures. There were three experimental conditions: individual-learning, paired-learning with an unfamiliar peer, and paired-learning with a familiar peer. ANCOVA results found that students paired with a familiar peer reported higher motivation in learning and more self-monitoring behaviors than those paired with an unfamiliar peer or who learned alone. Furthermore, students paired with both unfamiliar or familiar peers demonstrated better learning transfer than those who learned alone. Together, these results confirm the benefits of and support learning alongside a familiar peer during video lectures